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Predicting categories of improvement: 

The role of the working alliance 

The study represents our continuing refinement of the prediction of therapeutic change 
among imiversity counseling center clients. This analysis (as was the case with its predecessors, 
which were presented at the previous two AERA annual meetings) begins with the assumption 
that a basic question to which most clients want and deserve an answer prior to entering into 
coimseling is, “Am I going to get better?” The answer to this question is a categorical one, and in 
order to answer such a question, coimselors need their own answers to questions regarding “What 
kinds of outcomes are likely with what types of clients?” For both clients and coimselors, the 
answers to these questions are probabilistic in nature, communicating the likelihood of 
improvement or the likelihoods of various categories of improvement or types of outcome. 

Hummel (1995) proposed a rationale and framework for examining counseling outcomes 
in terms of categorical dependent variables. In that paper he argued the merit of focusing research 
attention on how independent variables affect the probability that a particular client would have a 
particular outcome or be in a particular outcome category (e.g., “greatly improved,” “improvedj” 
“unchanged,” “deteriorated”). At the 2000 annual meeting of the AERA, we reported on an 
analysis of the outcomes of 181 1 counseling clients seen at 38 different U.S. college and 
university counseling centers (Lichtenberg & Hummel, 2001). Client intake information (e.g., age, 
gender, ethnicity, readiness for counseling, symptom severity, history of previous counseling, 
current medication) — ^the limited sort of information that counselors might have on which to base 
an answer to a client’s question regarding the likelihood of improvement prior to the client’s 
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decision to enter counseling — was used to predicted client outcome. The outcome variable in this 
study was a dichotomous variable rimproved = a reduction in client symptoms; unimproved = 
lack of change or an increase in client symptoms). 

In the discussion following that presentation, it was suggested that outcome findings 
might be enhanced by considering (a) additional categories of improvement, (b) whether or not 
the clients entered counseling at a “clinical level” of distress, (c) the reliability of client change, 
and (d) the length of counseling (number of sessions) received by the client. Following these 
suggestions, a reanalysis of the counseling center data was undertaken and presented at last 
year’s annual meeting (Hummel & Lichtenberg, 2001). The goal was to build upon and further 
clarify and enhance the prediction of clients’ probabilities of various categories of counseling 
outcome based on those earlier suggestions. An ordinal logistic regression analysis was used to 
estimate the probabilities of four categories of client improvement: (a) some degree of reliable 
negative change (i.e., the client became worse), (b) no reliable change (i.e., the change could not be 
reliably distinguished from measurement error), (c) small to moderate reliable positive change 
(i.e., the client became better); and (d) large positive reliable [“clinically significant”] change (i.e., 
the client was much better; ES >.80); For the analysis, we used six independent variables. Four 
of these variables were those that had been found to be significant predictors of outcome in the 
2000 analysis; two were included as a result of discussions about that previous study. Five of 
these variables were based on information that a. counselor meeting a client for the first time 
would (or could) reasonably have available prior to a client’s commitment to counseling. The 
results of the analysis showed that previous experience as a client, readiness to change, level of 
symptomatic and interpersonal distress, pre-counseling clinical status, and the number of 
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counseling sessions clients in which a client might be involved were significantly related to 
probabilities of outcome category membership. 

The purpose of this study was to reanalyze the national counseling center data set with 
the goal of exploring the role o f process variables in the prediction of clients’ probabilities of 
various categories of counseling outcome. Specifically, the study focused on the contribution (if 
any) of the counselor-client working alliance to enh^cing our earlier predictions regarding 
therapy outcome category membership. 

Method 

Instruments 

Outcome Ouestionnaire-45 (version 2) (OQ-45; Lambert, Hansen, Umpress, Lunnen, 
Okiishi, Burglingame, & Reisinger, 1999). Pre- and post-counseling data were derived from the 
Outcome Ouestionnaire-45 . The OQ-45 is designed to measure client progress in therapy along 
three dimensions conceptualized by Lambert (1983) as important aspects of a client’s life: (a) 
subjective discomfort (i.e., how a person feels inside), (b) interpersonal relationships (i.e., how a 
person gets along with significant others), and (c) social role performance (i.e., how they manage 
in important life tasks such as work and school). 

Stapes of Change Scale (SCS; McConnaughy, Prochaska & Velcier, 1983). The Stages of 
Change Scale (SCS) is designed to assess clients’ readiness for change and their readiness to enter 
into and benefit from counseling. The scale assesses clients in terrns of Prochaska’ s stages of 
cheinge—stages at which clients might enter and begin counseling: Precontemplation . 
Contemplation . Action . Maintenance . 

Working Alliance Inventory (WAJ; Horvath & Greenberg, 1986, 1989). The WAJ was 
developed to measure the components of the therapeutic working allieince of Bordin’s (1979) 
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working alliance model. Two forms of the WAI, a client version (WAI-C) and a therapist version 
(WAI-T) were used in our analyses. Both versions contain three subscales: therapist and client 
agreement on therapeutic tasks, agreement on therapeutic goals , and the development of the bond 
between the therapist and client. 

Data Sources 

Participants in this study (n=862) were a subset of the 4679 clients participating in a 
nationwide study of college and university counseling centers conducted by the Research 
Consortium of Counseling and Psychological Services in Higher Education. The clients and 
counselors whose data were analyzed were those clients on whom we had pre- and post- 
counseling OQ45 scores and at least 2 complete scale scores among the 6 WAI subscales 
(Therapist and Client: Task, Goal, Bond). Missing values for WAI scores and other variables 
were estimated using the SPSS 10.0 expectation maximization (EM) algorithm and imputed into 
the data set for inclusion in our analyses. (Note: In all, 8.35% of the values in the data matrix 
were missing and imputed using this procedure; 16.63% of the WAI scales scores were missing; 
and among counselors 20.59% of the WAI scales scores were missing. Had missing values not 
been imputed, 42.1 1% of the sample would have been lost.) 

Of our client sample, approximately one third were male. The majority (75.1%) of clients 
were Caucasian. The counselors who saw the clients included practicum students, interns, and 
professional staff of the centers and represented a variety of fields, although most were 
counseling psychologists. They represented diverse ethnic backgrounds, although most were 
Caucasian. The majority were female, although a significant minority of the counselors were male. 
Approximately half of the counselors were student trainees, the majority of whom were 
counseling center interns, although a significant minority were practicum students. 

6 
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The data for this study were provided by the Research Consortium of Counseling and 
Psychological Services in Higher Education, which was established in 1990 to further research 
efforts on the practices of college and university counseling centers and the concerns of their 
clients. The data analyzed were collected over the period of 1997 and 1998 and represent a 
subset of the data available from the consortium. 

Analyses 

Using their intake (pre-counseling) scores on the OQ-45, clients were designated as either 
“clinical” or “non-clinical” at the onset of their counseling. In accordance with the OQ-45 manual, 
clients with total scores greater than or equal to 63 were designated as “clinical.” Intake and post- 
counseling ratings on the OQ-45 were used to calculate a categorical index of improvement. The 
ordered classification of four categories of client improvement (our dependent variable) was based 
on the OQ-45 ’s “reliable change index” (RCI) (Jacobson, Follette, & Revenstorf, 1984) as 
reported in the instrument’s manual (RCI =14). We used the RCI to construct an approximately 
90% confidence interval on the true change score (observed change score plus or minus 14). Our 
four categories of improvement were: (1) some degree of negative change, ( 2 ) no statistically 
significant change . .small to moderate positive change : and (4) large positive [“clinically 
significant”] change ( effect size > .80). 

An ordinal logistic regression' was used to estimate probabilities of categories of 
improvement as a function of the same variables found to be significant predictors in our most 
recent prediction model, i.e., clients’ intake scores on the OQ-45 social and interpersonal 
functioning scales and on the Contemplation subscale of the SCS and their history of previous 

' The ordinal logistic regression results were obtained using both MINTTAB and SPSS. The results agreed perfectly 
except in one respect. While the weights for the independent variables were the same in absolute value, in every case 
they had opposite signs. Using the weights from both MINITAB and SPSS to estimate probabilities of category 
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counseling, length of counseling completed by the client, and their clinical status at intake 
(clinical/nonclinical), plus the addition of the therapist and client WAI subscales. 

Results 

Using the four categories defined above as the dependent measure and the independent 
variables from the previous year’s study (i.e., client intake scores on the OQ-45 social and 
interpersonal functioning scales and on the Contemplation subscale of the SCS and their history 
of previous counseling, length of counseling completed by the client, and their clinical status at 
intake), a ordinal logistic regression was run but without inclusion of the process measures (i.e., 
the therapist and client WAI ratings). The result was highly significant, (6) = 182.349, 
p<1.0*E-307. The results of our analysis of clients’ ratings of the working alliance alone as 
predictors of outcome category were non-significant, (3) = .779, p=.855. The addition of the 
clients’ WAI ratings did not significantly enhance the prediction over that of the model without 
the inclusion of the clients’ WAI ratings, (3) = .721, p=.868. The result of our analysis of 
therapists’ ratings of the working alliance alone as predictors of outcome category was 
significant, y^ (3) = 15.973, p=.001. The addition of the therapists’ WAI ratings significantly 
enhanced the prediction over that of the model without the inclusion of the therapists’ WAI 
ratings, y^ (3) = 10.562, p=.014. At this point, the full model contained nine variables, six that 
were found in earlier research to be significant, and the therapist’s three WAI scales, y^ (9) = 
192.911, p<l .0*E-307. However, the weights of two predictors were non-significant in this 
model. Since it was our intent to build a simulation model to see how probabilities of category 
membership changed as function of varying predictor values, we did not want to manipulate the 
values of non-significant variables. For this reason, two variables, clinical/non-clinical and 

membership, it became clear to us that, given our way of evaluating the cumulative logistic distribution, (1+e '^) ', 
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therapist’s WAI Goal Scale, were eliminated from the model. The final, then, was based on seven 
variables, (7) = 187.549, p<1.0*E-307. 

The following table presents the constants and weights used to compute the probabilities 
of category membership. When the means for the predictors are used in computing the 
probabilities of category membership, the following probabilities result: .069, .507, .302, and 
.122. These can be compared to actual proportion of clients in each category: .096, .478, .275, 
and. 151.^ 



Logistic Regression 
Tabie 










Predictor 


Coef 


StDev 


Z 


P 


Const(1) 


2.04760 


0.978400 


2.09 


0.036 


Const(2) 


4.95060 


0.991400 


4.99 


0.000 


Const(3) 


6.62100 


1.002000 


6.61 


0.000 


Previous 

Counseling 


0.29230 


0.133700 


2.19 


0.029 


Number of Sessions 


-0.04855 


0.016030 


-3.03 


0.002 


Symptom 

Discomfort 


-0.04155 


0.004914 


-8.45 


0.000 


Interpersonal 


-0.03209 


0.011420 


-2.81 


o;oo5 


Relationships 

Contemplation 


-0.04477 


0.018630 


-2.40 


0.016 


Therapist WAI Task 


0.02852 


0.013100 


2.18 


0.030 


Therapist WAI Bond 


-0.03893 


0.014560. 


-2.67 


0.007 



Care must be taken when interpreting these weights. While the signs of the weights are 
correct, the weights’ influence on the probabilities of category membership are the opposite of 
what would be expected based on those signs. Having had previous counseling and increases in 
Therapist WAI Task are associated with decreases in the probability of a positive outcome. 



the signs of the SPSS weights were in error. 

^ Had the incorrect weight signs in the SPSS results been used (see Footnote One), the probabilities of category 
membership would bear no resemblance to the to actual proportion of clients in each category. 
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Increases in the remaining predictors, all of which have negative signs, are associated with 
increases in the probability of a positive outcome. 

The fact that Therapist WAI Task and Therapist WAI Bond have different signs in the 
model is counterintuitive, especially since for these 862 clients they correlate .701. The situation 
is clarified by viewing Therapist WAI Task as an error suppressor, in this case, a variable that 
correlates with another predictor but has essentially no relationship to the criterion. 

The following figure presents the interactive spreadsheet model that we use to simulate 
the probabilities of category membership for various types of clients. The model uses common 
controls, e.g., a drop-down list box and spinners, to set the values of the predictors. Several 
features are not obvious and deserve explanation. “Clinical” has no spinner because it was 
dropped from the model. However, if Symptom Discomfort and Interpersonal Relations are high 
enough or low enough, then the client maybe deemed “clinical” or not regardless of the value of 
Social Role Performance, which is not in the model. A note next to “Therapist Task” points out 
that it is an “Error suppressor slaved to Therapist Bond.” This means that if “Therapist Bond” 
is changed, then “Therapist Task” is set to the predicted value of “Therapist Task” given 
“Therapist Bond.” Further, “Therapist Task” cannot be set to a value higher than the lesser of 
the predicted value plus (approximately) two standard errors of the estimate or 84. Likewise, 
“Therapist Task” cannot be set to a value lower than the greater of the predicted value minus 
(approximately) two standard errors of the estimate or 12.^ 



Insert Figure 1 about here 



^ A copy of the interactive model may be obtained from either author. The model requires Microsoft Exel and 
macros must be enabled. 
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Discussion 

Our previous studies demonstrated that the likelihood of various therapy outcomes (i.e., 
negative change, non-significant change, small to moderate positive change, large positive 
[“clinically significant”] change) could be predicted given only basic information that a counselor 
meeting a client for the first time would (or could) reasonably have available prior to a client’s 
commitment to counseling. The results of the present study demonstrate that adding to those 
predictors information regarding perceptions of the quality of the unfolding process of 
counseling — specifically, the counselor’s view of the working alliance — can significantly enhance 
those predictions. That therapist, rather than client, perceptions of the working alliance proved 
to be an additional predictor seems to be at odds with previous literature (e.g., Horvath, 1994; 
Horvath & Greenberg, 1986; Horvath & Symonds, 1991) that suggests that client, rather than 
therapist, ratings of the working alliance are more strongly related to therapeutic outcome ratings. 

Counseling centers, like other behavioral healthcare settings, need to be able to meet the 
primary goals of health care, namely, to help the client when possible and to do no harm (Lyons, 
Howard, O’ Mahoney & Lish, 1997; Ogles, Lambert & Masters, 1996; Sederer, Dickey & 
Hermann, 1996). Meeting these goals requires being able to establish and communicate realistic 
probabilities for various counseling outcomes for. the individual clients that are seen by their 
counseling staff so that these clients can make informed decisions about entering into counseling. 
Awareness of those variables that are predictive of various therapy outcomes will help 
counselors provide the information that clients need to make such decisions. 
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